Data communication networks such as the Internet, Wide Area Networks (WANs) and Local Area Networks (LANs), offer tremendously efficient means of organizing and distributing computerized data. These efficiencies have resulted in their widespread use for both business and personal applications. For example, the Internet is now commonly used to support database services across broad geographical areas—Internet banking, centralized office services and many other similar services are examples of systems which employ a single central database and multiple application servers, accessible over the Internet.
FIG. 1 presents an exemplary block diagram of such a database system 10. In this example, multiple copies or “instances” of a software application are operating on multiple servers or processors 12, 12′. Each of the application instances must access a main database 14 to obtain data, and store data to the main database 14 so that other application instances can access current versions of the shared data. End users employ client computers 16 to access the application instances on the multiple servers or processors 12, 12′. The communications between the client computers 16 and the multiple servers or processors 12, 12′ may be done over the Internet or some other communication network, as may the communications between the multiple servers or processors 12, 12′ and the main database 14.
As will be explained in greater detail hereinafter, in current versions of such systems where the caches must be kept coherent, the multiple instances of the software application, as well as the multiple servers or processors 12, 12′ are usually homogeneous—employing the same structures, organization, protocols and store mechanisms. This homogeneity makes communication between the components of the system very easy to do.
Operation of such a system generally proceeds as follows: if, for example, an end user wishes to pay bills or transfer money from one bank account to another, he may use his client computer 16 to access an instance of a banking software application on server 12. The banking software application, in turn, retrieves data from the main database 14 and returns new data records resulting from end user transactions, to the main database 14. Clearly, the accessing and changing of data on the main database 14 must be properly coordinated so that all instances of the banking software application are handling “current” versions of the data.
It is also clear that a lot of resources can be consumed if all instances of a software application are continuously contacting the main database 14 in response to every access requirement or change in data. To avoid a massive number of communications which would be a huge burden on system resources, each instance of the software application typically uses a local cache 18, 18′ to store data. With a local cache system, an instance of the software application can access the main database 14 to obtain a set of data which it stores in its local cache 18, 18′. While processing is performed by the instance of the software application, changes are made to the data in the local cache 18, 18′ and the main database 14 is updated when all of the changes have been committed. Since the contents of the local caches 18, 18′ are consistent with the data stored in the main database 14, the application instance may continue to access the data from the cache 18, 18′ in lieu of going to the database 14.
In the case of Internet banking, for example, an instance of the software application on server 12′ may read a summary sheet for a particular client from the main database 14, including account numbers, balances on each account, and a record of recent transactions, storing it on the local cache 18′. Rather than requesting this data from the main database 14 each time a client accesses information from the summary sheet, the instance application 12′ accesses the information in its cache.
The difficulty is that if the application on server 12 performs a similar operation and loads the summary sheet for the same client into its local cache 18, and then makes a change that subsequently gets written back to the main database 14 then the contents of the local cache 18′ are rendered stale and inconsistent with the data in the main database. There must be an additional mechanism to ensure that local cache 18 becomes aware that its state is no longer consistent. This is known as the “cache coherency” problem.
There are a number of mechanisms used in the art to ensure “data coherency” while processing on such systems; that is, ensuring that all instances of the software application are using current, accurate cached data. Two such mechanisms are “pessimistic locking” and “optimistic locking”.
“Pessimistic locking” is a technique by which data on the main database 14 is “locked” as soon as it is accessed to be updated. Once the data to be updated has been locked, the instance of the software application that accessed the data can make the required changes, and then either “commit” or “rollback”—after which the lock is released. “Committing” means that the main database 14 is instructed to accept the new data, while “rollback” means that the changes have been rejected and the data is to be restored to its original state. If any other instance of the software application attempts to acquire a lock of the same data while it is being processed by an instance of the software application, they will be forced to wait until the earlier transaction has been completed.
This approach is called “pessimistic” because it presupposes that other instances of a software application will try to change a data record while a first instance is modifying it.
Pessimistic locking has at least three major problems:    1. “lockout”, where one instance of a software application selects a record for update, and then fails to finish or abort the transaction. This locks out all other instances of the software application from accessing the locked out data;    2. “deadlock”, where two instances of the software application are awaiting the release of locks held by the other. Say, for example, that Instance A and Instance B are both updating the main database 14 at the same time, and Instance A locks a record, then attempts to acquire a lock held by Instance B. If Instance B is waiting to obtain a lock held by Instance A, then neither transaction can be completed, and the two Instances will be “deadlocked”; and    3. performance degradation from requiring the software application on every server to access the data from the main database. This puts the application in the position of not being able to make reasonable use of its own local cache.
In contrast, “optimistic locking” does not lock records when they are read, but proceeds on the assumption that data being processed has not changed since it was read. Rather than taking out locks, an instance of a software application simply checks to see whether a data record has been updated since it obtained the data for editing. This approach is particularly useful in a stateless environment such as the World Wide Web. As well, because no locks are used, there are no lockout or deadlock problems.
However, optimistic locking does not provide “concurrency control”. Concurrency control is a mechanism which ensures that data being stored on the main database 14 is consistent with the data that was read from it in the first place; that is, that no other transaction or instance of a software application has updated the data on the main database 14 after it was read.
Various methods of ensuring data consistency are known in the art. One simple technique is to read a “key value” along with a data record, and then write that same key value back to the main database 14 when the data record is updated. The main database 14 then changes the key value after a write so that other transactions can tell that the record was changed (i.e. by comparing the key value they hold, to the one currently associated with the data record). Should any other transaction or application instance attempt to update the record with the old (and now obsolete) key value, the database will recognize the key value as obsolete and reject the update.
Having addressed these major issues, Internet-based database technology is generally effective and technologically mature. However, the challenge of providing a distributed-application database system rises tremendously when the computers, the software or other components in the system, are different from one another.
Typically, the databases, caches and instances of software applications and other supporting software used in such systems are homogenous. That is, the databases and caches maintain the same structure and organization as one another, and all the protocols and store mechanisms are understood by all of the participants. It is much more difficult to coordinate the data between disparate applications running on differing hardware and operating system platforms.
An effective strategy for coordinating data between disparate applications running on differing hardware and operating system platforms has not yet been developed. There is therefore a need for a means of coordinating data over disparate computer systems, provided with consideration for the problems outlined above.